Europhysics Letters 



PREPRINT 



Temporal Effects of Agent Aggregation in the Dynamics 
of a Competing Population 

C. H. Yeung and K. Y. Michael Wong 

Department of Physics, Hong Kong University of Science and Technology, Clear Water 
Bay, Hong Kong, China 



PACS. 02.50.-r - Probability theory, stochastic processes, and statistics. 

PACS. 05. 40. -a - Fluctuation phenomena, random processes, noise and Brownian motion. 

PACS. 89. 20. -a - Interdisciplinary applications of physics. 



Abstract. - We propose a model of a competing population whose agents have a tendency to 
balance their decisions in time. We find phase transitions to oscillatory behavior, explainable by 
the aggregation of agents into two groups. On a longer time scale, we find that the aggregation 
of smart agents is able to explain the lifetime distribution of epochs to 8 decades of probability. 



Many natural and artificial systems involve interacting agents, each making independent 
decisions to pursue their own short-term objectives, but globally exhibiting long-time correla- 
tions beyond the memory sizes or the action cycles of the individuals [1-3]. Examples include 
the price fluctuations in financial markets [4], traffic patterns on the Internet [5], and the 
lifespan and metabolic time scales of living organisms. It is interesting to consider the extent 
to which the inherent properties of the systems can affect the temporal behavior. Correla- 
tions beyond individual memory sizes can be consequences of interaction among agents. For 
example, some agents may find it advantageous to correlate their decisions over an extended 
period of time, leading to the emergence of epochs dominated by a group. This constitutes 
a system with multiple time scales. On a long time scale, it is described by the ages of the 
epochs, besides the shorter time scale of the underlying dynamics (usually determined by the 
memory sizes or the action cycles of the agents). 

In this Letter, we analyze a competing population model with the above features. The 
model is most applicable to financial markets in which agents trade with finite capital, but 
can be extended to other multiagent systems such as routers in communication networks 
attempting to transmit multiclass traffic in a fair way. In an effort to avoid overdrawing 
her capital, an agent making a buying decision at an instant is more inclined to make a 
subsequent selling decision, and vice versa. This creates a natural short-time scale for the 
buy-and-sell cycles of the market. To model the adaptive attributes of the agents, each agent 
can occasionally revise her decision by following some adaptive strategy she perceives to be 
successful, analogous to the Minority Game (MG) [3]. We are interested in studying how 
agents aggregate in their decision making process in the short as well as long time scales. 

Specifically, we consider a population of N agents competing to maximize their individual 
utility, while striving to maintain a balance of their capital, N being odd. At each time step 
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t, agent i makes a bid bi(t) = 1 or 0, and the minority group among the N agents wins. "1" 
and "0" may represent "buy" and "sell" decisions in a financial market. To model agents 
with concerns for both finite capital and adatpive strategies, agent i makes buying bids with 
probabilities P[bi(t) — 1], where 

P[h(t) = 1] = kf(d(t)) + (1 - Strategy = 1], (1) 

where < k < 1 is the restoring strength determining the statistical weights of the two factors 
in Eq. (JTJ. 

The first term models the consideration of finite capital in an agent's buying decisions at 
time t, whose probabilities increase with her capital Ci(t) at the instant. For example, we 
choose in this Letter a three-step function f(ci(t)) which is equal to 0, 1/2 and 1 respectively 
in the ranges < Cj(t) < c <7 c< < Ci(t) < c>, and c> < Ci(t) < 1. For a buying bid, the 
capital reduces by (1 — a)ci(t), in exchange for the same value of stocks (to the lowest order 
approximation, we have neglected price changes due to fluctuations in supply and demand). 
Similarly, for a selling bid, the volume of stocks reduces by (1 — a) [1 — Cj(f)], in exchange for 
the same value of capital. Thus, the capital change is given by 

d(t + 1) = a Cl (t) + (l - )[1 - h{t)]. (2) 

In Eq. J5J, the second term counts the number of selling bids in history at a value of 1 — a, 
and the first term shows that its effect on the capital is discounted exponentially at a rate of 
a per step. Hence, a is referred to as the restoring memory factor. Small a corresponds to 
short-term restoring, namely, balancing buying and selling in small number of steps, whereas 
large a corresponds to long-term restoring. When there are more selling (buying) bids in 
the recent bid history, the capital tends to be high (low). Through the probability f(a(t)), 
the agent tends to balance the more frequent recent bids with the opposite actions. This is 
equivalent to adding a restoring force to the decision making process of the agents. 

The second term models an agent's tendency to use adaptive strategies, and the proba- 
bility Pstrategy [&» (t) = 1] can be determined by standard games such as the MG [3] or the 
Evolutionary Minority Game (EMG) [6]. In MG, strategies are binary functions mapping an 
m-bit signal to an output bit, the signal here being the winning bits of the most recent m 
steps. Before the game starts, each agent randomly picks s strategies. At each time step, 
the cumulative payoffs of strategies that give a correct prediction increase by 1, while those 
predicting incorrectly decrease by 1. Each agent then follows the strategy with the highest 
cumulative payoff among those she owns. The EMG differs from the MG in details, but shares 
the same feature that the adopted strategies evolve according to historical performances. 

The system behaviour is characterized by the variance of buyers at the steady state. In 
the simulations, all agents initially hold equal value of capital and stocks, and the cumulative 
payoffs of strategies in MG are randomized with variance R [7]. Remarkably, Fig. 1(a) shows 
that the system behaviour crosses over from a strategy- specific regime at k — to a memory- 
factor- specific regime when fc approaches 1. In the latter regime, the variances approach values 
dependent on a, irrespective of the type of the adaptive strategies (such as in MG or EMG). 
Thus in the strongly restoring limit, the system behavior remains the same for a given value 
of a, as long as some adaptive strategy is adopted. For large a in the strongly restoring limit, 
the restoring force tends to bring the variance of the system to the coin-toss limit a 2 /N = 1/4, 
implying that the agents make random and uncorrelated decisions. For small a in the same 
limit, the variance has a value lower than the coin-toss limit. Note that the variances in the 
limit fc approaching 1 jumps discontinuously to the coin-toss limit at fe=l (that is, when the 
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Fig. 1 - (a) The variance as a function of the restoring strength k (N — 101, 50,000 steps, 1,000 
samples; for MG, R = 51, s = 2). (b) The variance as a function of the restoring memory factor a at 
k — 0.99 and a c = 2/3. Inset: The distribution of capital for different a indicated in the legend. 



decisions are solely based on the concern for finite capital), showing that adaptive strategies, 
even only rarely used by the agents, are essential in bringing the system to an efficient state. 

Figure 1(b) shows the variance in the strongly restoring limit, which increases with the 
restoring memory factor a. It undergoes a continuous phase transition to the coin-toss limit 
at a critical a, whose value is independent of the type of adaptive strategies. The nature 
of this transition is revealed in the distribution of capital at a steady-state instant of the 
system, which has two sharp peaks in the short-term memory phase, but one otherwise (Fig. 
1(b) inset). The two-peaked distribution arises from the intentions of the agents to balance 
their budget in a small number of steps, causing them to aggregate into two groups, A and 
B, analogous to the crowd- anticrowd picture of multiagent systems [8]. Group A consists 
of agents making alternating buy and sell bids at odd and even time steps respectively, and 
agents in group B make opposite alternating bids. This self-organization of agent behavior 
is further confirmed by the spectrum of a Fast Fourier Transform of the capital of a typical 
agent, which shows a sharp peak at the frequency 0.5. In fact, one can show from Eq. Q that 
when a < a c = min[c</(l — c<), (1 — c>)/c>], a limit cycle of period 2 can be formed from 
the alternating bids of an agent. This enables the capital of an agent to avoid staying in the 
intermediate region (c<, c>) of random bids, resulting in a segregated phase with two groups 
of agents. The value of a c coincides with the phase transition point in Fig. 1(b). In contrast, 
for a > a c , there appears a growing fraction of agents in the region of random bids, and the 
period-2 dynamics is not sustained, resulting in a clustered phase with a coin-tossed variance. 
We note in passing that segregated and clustered behavior have been observed in a model of 
EMG due to a different cause [9] , but the issue of time scales has not been studied. 

We now consider temporal effects beyond the period-2 oscillations, focusing on the limit 
k approaching 1 in the segregated phase. At each time step, an average of (1 — k)N = k 
agents make decisions according to adaptive strategies, and thus have a chance to switch from 
group A to B, or vice versa. Then, groups A and B change from majority to minority, or vice 
versa, in a time scale longer than the period-2 oscillations. The emergence of these multiple 
time scales is illustrated in Fig. 2, in which the buyer population is essentially oscillating 
with pcriod-2. Occasional phase slips in the buyer population signify switches from group A 
being the winners to group B, or vice versa. This corresponds to instants in Fig. 2 where the 
population of group A crosses the minority-majority boundary. The lifetime of an epoch is 
the duration for which group A or B remains winning continuously. 

To analyze the variance and the epoch lifetimes, we note that since the history is dominated 
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Fig. 2 - The population size of the buyers and group A for a particular sample (N = 101, m = 1, 
s = 2, R = 51 , k = 0.99, a = 0.4, 10,000 steps). 



by period-2 oscillations, there is a reduction in the effective dimensionality of the dynamics. 
This greatly simplifies the analysis of the dynamics, rendering it generally applicable to MGs 
of arbitrary memory size to. Indeed, the two most common signals received by the agents are 
the series of alternating "0" and "1" (e.g. "0101" and "1010" in the case of m = 4), referred to 
as the epoch steady signals (ES signals). After the first to — 1 steps since their birth, the epochs 
enter the epoch steady state (ES state), in which the switching of agents between groups A 
and B are determined by their responses to the ES signals. Thus, there are 4 categories of 
strategies according to their response to the ES signals: followers, whose decision follows the 
last bit of the ES signals; sellers, whose decision is 0; buyers, whose decision is 1; contrarian, 
whose decision is opposite to the last bit of the ES signals. 

For small k such that (L) > m, the ES signals are the most frequent ones, causing the 
contrarian strategies to acquire the highest cumulative payoffs, and the follower strategies the 
lowest. The buyer and seller strategies predict correctly half of the times in the ES state, 
and their cumulative payoffs arc intermediate. At the ES state, the agents making minority 
strategic decisions are those holding at least one contrarian strategy. They are referred to as 
smart agents. The probability of finding a smart agent is 1 — (3/4) s . Otherwise, if agents have 
at least one of the buyer or seller strategy, their strategic bid follows the minority decision in 
the ES state with probability 1/2, and are referred to as mediocre agents. The probability of 
this case is (3/4) s — (l/4) s . With probability (l/4) s , the agents only have follower strategies at 
hand, and their strategic bid follows the majority decision in the ES state. They are referred 
to as the dump agents. On average, the probability of an agent bidding for the minority 
decision in the ES state is T ES = 1 - (3/4) s /2 - (l/4) s /2. 

The analysis of the first m — 1 steps, referred to as the epoch transient state (ET state), 
is more involved. In this case, the smart agents are no longer receiving the ES signals that 
they are most adapted to. In particular, for those smart agents who hold only one contrarian 
strategy, the payoff of the contrarian strategy is accumulated owing to its successful responses 
to the dominant ES signals, irrespective of their responses to the ET signals. Thus, their 
average probability of making a minority decision is 1/2. On the other hand, for those smart 
agents who hold more than one contrarian strategies, the payoffs of the contrarian strategies 
are accumulated to the same extent in the ES state, and the ET signals are able to favour 
the payoff accumulation of those contrarian strategies that predict the correct minority bit in 
the ES state. These agents, who adapt to the ES state as well as partially to the ET state, 
are referred to as the super-smart agents. Their probability of making a minority decision is 
Tsuper, whose average value can be shown to be greater then 1/2 [10], using a self-consistent 
and semi-empirical approach through an analysis of the historical occurrence of signals, or 
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an approximation that all ET signals have the same probability of occurrence. Similarly, 
there are super-mediocre agents who hold more than one buyer or seller strategies. Since 
all dump agents hold more than one follower strategies, they can also be called super-dump 
agents. Their average probability of making a minority decision is also T sup cr, in contrast to 
the probability of 1/2 for the rest of the population. On average, the probability of an agent 
bidding for the minority decision in the ET state is T E x = /super r supcr + (1 — /super)/2, where 
/super = l-( S /4)(3/4) s - 1 -( S /2)(l/4)- 1 . 

We first analyze the dynamics in the mean-rate approximation. The average probability 
that an agent uses an adaptive strategy and switches side at time t is, respectively, (1 — k)T{t) 
from majority to minority, and (1 — k)(l — T(tj) in the opposite direction, where T(t) = T E s 
and Tet in the ES and ET states respectively. The master equation for the distribution of 
the number N A of agents in group A can then be solved numerically or by Monte Carlo. 

As shown in Fig. 3, the average epoch lifetime (L) decreases with the average number k of 
agents using adaptive strategies. When k approaches 0, (L) appears to approach the expected 
result of (L) ~ fc _1 . This implies that during the average lifetime of an epoch, an average of 
k{L) w 5 agents make strategic decisions which may result in switching sides. Furthermore, 
the prediction of the mean-rate approximation has an excellent agreement with the simulation 
results. A similarly excellent agreement is shown in Fig. 3 inset for the variance a 2 . When 
k approaches 0, the master equation for P(N A ) can be approximated by a Markov chain, 
yielding P(N A ) ~ [(1 - TbsV^s] 1 ^^^ 721 and a variance of 2[(1 - T ES )/(2T ES - l)] 2 + 
2[(1 - T ES )/(2T ES - 1)] + 1/4. 

Figure 4 shows that the epoch lifetimes follow an exponential distribution near its max- 
imum at small L. The exponential decay rate depends on k and is insensitive to N. The 
mean-rate approximation has an excellent agreement with simulation results in this regime. 
When L increases further, the distribution crosses over to an exponential one with a lower 
decay rate dependent on both k and N. However, the mean-rate approximation predicts a 
lower distribution of long epochs than simulations. As shown in Fig. 4 inset, the decay rates of 
both the simulation and the mean-rate approximation approaches the same asymptotic value 
for large N, but their differences are significant in the mesoscopic regime. Analysis shows that 
the differences scale as jv~ 11±01 . 

The discrepancy arises from the inability of the mean-rate approximation to differentiate 
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Fig. 4 - Lifetime distribution (s = 2, R = 51, k = 0.99, a = 0.4, 30,000 steps with 30,000 samples) 
compared with the mean-rate and multirate approximations. Inset: decay rate of long epochs as a 
function of N for m = 1. 

the aggregation effects of various agents in the majority and minority groups. Suppose it 
happens that the majority group in an epoch consists of less smart agents than the minority 
group. It follows that when an agent in the majority group chooses to make a strategic 
decision, it is less likely that she switches to the minority group. The converse is valid for 
dump agents, but to a lesser extent due to their smaller population. This reduces the flow of 
agents from the majority to the minority group, and hence lengthen the lifespan of an epoch. 
This effect is most significant in the mesoscopic regime when the fluctuations of the smart 
agent aggregate and the minority group size are important. 

The evidence that smart agents aggregate in the minority group at the expense of the 
majority group is demonstrated in Fig. 5. We measure the difference in the number of smart 
agents between the minority and majority groups (-/V m j n , s — iV ma j ;S ) averaged over an epoch. 
Compared with the distribution for short epochs, the peak of the distribution for long epochs 
is shifted to the positive side. This shows that long epochs have more smart agents distributed 
in the minority group on average. 

This leads to the second approximation of our analysis, the multirate approximation. We 
denote by Na s , ^Am and AAd the number of smart, mediocre and dump agents in group 
A respectively. Instead of using a single average probability of switching from the majority 
to the minority group for all agents and vice versa, the average transition probabilities for 
each of these agent types is taken into account. The master equation for the probability 
P(Nas, Naui, -^Ad) can be solved by Monte Carlo. As shown in Fig. 4, the multirate approxi- 
mation yields a significantly higher probability for long epochs compared with the mean-rate 
approximation, resulting in an excellent agreement with simulation results over 8 decades of 
probability. Furthermore, the distribution of smart agents derived from the multirate approx- 
imation has an excellent agreement with simulation results, as shown in Fig. 5. This shows 
that the aggregation of smart agents is crucial in explaining the occurrence of long epochs. 

Due to their generic nature, effects of smart agent aggregation are expected to be present 
for adaptive strategies other than MG strategies. The essence is the stabilization of the 
minority group by the smart agents, whose strategies are favored by the sustenance of an 
epoch. Note that the smart agents do not have any intrinsic preference to either group A or 
B; they favor the group they aggregate in as long as it is the minority. Differences among the 
various adaptive strategies only come from details of the transition rate between the majority 
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Fig. 5 - Distribution of the difference (iV m in,s — AT ma j, B ) between the number of smart agents in the 
minority and majority groups (N = 51, m = 6, s = 2, R = 51, k = 0.51, a — 0.4, 30,000 steps with 
30,000 samples). The average lifetime L « 11. Epochs with short and long lifetimes are defined by 
L < 20 and L > 100 respectively. 

and minority groups. 

In summary, we have proposed a model of a competing population with multiple time 
scales. When the tendency to balance the decisions of the agents is strong, changing their 
intentions from long-term to short-term induces phase transitions to oscillations with two 
groups of agents making alternating but opposite decisions, and the behavior is independent 
of the details of the adaptive strategies. On a longer time scale, the history is characterized 
by epochs dominated by one of the two groups. Epochs end when sufficient numbers of 
agents follow adaptive strategies and switch to the winning side. Epochs are stabilized by the 
aggregation of smart agents on the winning side in the mesoscopic regime. The study of epoch 
lifetime is important to the agents who want to stay on the winning side as long as possible, 
or at least to have a knowledge of how long they would prevail. These generic features 
are relevant to multiagent systems such as financial markets of agents with finite capital, 
distributed control of multiclass traffic in communication networks, and other systems with 
competing aggregates of agents. 
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